03. Project Instructions

Semantic Segmentation

Introduction

In this project, you'll label the pixels of a road in images using a Fully Convolutional Network (FCN).

Udacity Workspace

We have made a Udacity GPU workspace available for use for this project. Note that you will want to toggle off the GPU when you are coding vs. training and testing, as there is a certain limited amount of hours given for the Nanodegree for the GPU, but the version without the GPU toggled off does not have a limit.

Workspace Tutorial

Never used a Udacity workspace before? Check out this lesson added into Term 1 for a brief primer.

Data location

There is a small, but significant difference between the location of the data used by default in the project repository, ./data (i.e. the local directory), and in the workspace - /data. If you are using the workspace, you'll want to keep the location as such and not create another data directory (we've removed the one included in the repository), so that your workspace can correctly find the directory containing both the road images and VGG model (you do not need to download these again within the workspace).

However, if you want to switch between your workspace and a local machine, or vice versa, you'll want to keep the location of these in mind.

Local/AWS Setup

1) You are required to use a GPU for this project - you can use AWS or run it locally if you have your own GPU.

2) Clone the repo from https://github.com/udacity/CarND-Semantic-Segmentation

Frameworks and Packages

3) Make sure you have the following is installed:

Dataset

4) Download the Kitti Road dataset from here. Extract the dataset in the data folder. This will create the folder data_road with all the training and test images.

Start

Implement

Implement the code in the main.py module indicated by the "TODO" comments.
The comments indicated with "OPTIONAL" tag are not required to complete.

Run

Run the following command to run the project:

python main.py

Note: If running this in Jupyter Notebook system messages, such as those regarding test status, may appear in the terminal rather than the notebook.

Submission

  1. Ensure you've passed all the unit tests.
  2. Ensure you pass all points on the rubric.
  3. When you're done with the project, please submit the following in a zip file.
  • helper.py
  • main.py
  • project_tests.py
  • Newest inference images from runs folder (all images from the most recent run)

Tips

  • The link for the frozen VGG16 model is hardcoded into helper.py. The model can be found here, but is already downloaded if using the Udacity workspace.
  • The model is not vanilla VGG16, but a fully convolutional version, which already contains the 1x1 convolutions to replace the fully connected layers. Please see this post for more information. A summary of additional points, follow.
  • The original FCN-8s was trained in stages. The authors later uploaded a version that was trained all at once to their GitHub repo. The version in the GitHub repo has one important difference: The outputs of pooling layers 3 and 4 are scaled before they are fed into the 1x1 convolutions. As a result, some students have found that the model learns much better with the scaling layers included. The model may not converge substantially faster, but may reach a higher IoU and accuracy.
  • When adding l2-regularization, setting a regularizer in the arguments of the tf.layers is not enough. Regularization loss terms must be manually added to your loss function. otherwise regularization is not implemented.
  • The VGG model may be larger than some you have used before, so be conscious of your memory usage - you may need to use fairly small batch sizes in order to avoid running out of memory when training.
  • The VGG model, as well as semantic segmentation as a whole, may be slower than previous, simpler models you have constructed. You may want to print additional information during training to be more aware of progress - depending on batch size and number of batches per epoch, each epoch could be 2-3 minutes or more even with the GPU.
  • Given the above, you may consider prototyping on just a subset of the data until you are sure your model works correctly!

Using GitHub and Creating Effective READMEs

If you are unfamiliar with GitHub , Udacity has a brief GitHub tutorial to get you started. Udacity also provides a more detailed free course on git and GitHub.

To learn about REAMDE files and Markdown, Udacity provides a free course on READMEs, as well.

GitHub also provides a tutorial about creating Markdown files.

Project Support

If you are stuck or having difficulties with the project, don't lose hope! Remember to talk to your mentor, ask (and answer!) questions on Knowledge tagged with the project name, and reach out to your fellow students and mentors in the Semantic Segmentation channel in your Student Hub. Don't forget to check out the project Q&A from the previous page as well!